• Compute-Efficient Deep Learning: Algorithmic Trends and Opportunities

    Updated: 2023-04-30 19:38:56
    : Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Compute-Efficient Deep Learning : Algorithmic Trends and Opportunities Brian R . Bartoldson , Bhavya Kailkhura , Davis Blalock 24(122 1 77, 2023. Abstract Although deep learning has made great progress in recent years , the exploding economic and environmental costs of training neural networks are becoming unsustainable . To address this problem , there has been a great deal of research on algorithmically-efficient deep learning which seeks to reduce training costs not at the hardware or implementation level , but through changes in the semantics of the training program . In this paper , we present

  • An Annotated Graph Model with Differential Degree Heterogeneity for Directed Networks

    Updated: 2023-04-30 19:38:56
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us An Annotated Graph Model with Differential Degree Heterogeneity for Directed Networks Stefan Stein , Chenlei Leng 24(119 1 69, 2023. Abstract Directed networks are conveniently represented as graphs in which ordered edges encode interactions between vertices . Despite their wide availability , there is a shortage of statistical models amenable for inference , specially when contextual information and degree heterogeneity are present . This paper presents an annotated graph model with parameters explicitly accounting for these features . To overcome the curse of dimensionality due to modelling degree

  • A Unified Framework for Optimization-Based Graph Coarsening

    Updated: 2023-04-30 19:38:56
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us A Unified Framework for Optimization-Based Graph Coarsening Manoj Kumar , Anurag Sharma , Sandeep Kumar 24(118 1 50, 2023. Abstract Graph coarsening is a widely used dimensionality reduction technique for approaching large-scale graph machine-learning problems . Given a large graph , graph coarsening aims to learn a smaller-tractable graph while preserving the properties of the originally given graph . Graph data consist of node features and graph matrix e.g . adjacency and Laplacian The existing graph coarsening methods ignore the node features and rely solely on a graph matrix to simplify graphs .

  • SQLFlow: An Extensible Toolkit Integrating DB and AI

    Updated: 2023-04-30 19:38:56
    Integrating AI algorithms into databases is an ongoing effort in both academia and industry. We introduce SQLFlow, a toolkit seamlessly combining data manipulations and AI operations that can be run locally or remotely. SQLFlow extends SQL syntax to support typical AI tasks including model training, inference, interpretation, and mathematical optimization. It is compatible with a variety of database management systems (DBMS) and AI engines, including MySQL, TiDB, MaxCompute, and Hive, as well as TensorFlow, scikit-learn, and XGBoost. Documentations and case studies are available at https://sqlflow.org. The source code and additional details can be found at https://github.com/sql-machine-learning/sqlflow.

  • Learning Good State and Action Representations for Markov Decision Process via Tensor Decomposition

    Updated: 2023-04-30 19:38:56
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Learning Good State and Action Representations for Markov Decision Process via Tensor Decomposition Chengzhuo Ni , Yaqi Duan , Munther Dahleh , Mengdi Wang , Anru R . Zhang 24(115 1 53, 2023. Abstract The transition kernel of a continuous-state-action Markov decision process MDP admits a natural tensor structure . This paper proposes a tensor-inspired unsupervised learning method to identify meaningful low-dimensional state and action representations from empirical trajectories . The method exploits the MDP's tensor structure by kernelization , importance sampling and low-Tucker-rank approximation .

  • The d-Separation Criterion in Categorical Probability

    Updated: 2023-04-30 19:38:56
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us The d-Separation Criterion in Categorical Probability Tobias Fritz , Andreas Klingler 24(46 1 49, 2023. Abstract The d-separation criterion detects the compatibility of a joint probability distribution with a directed acyclic graph through certain conditional independences . In this work , we study this problem in the context of categorical probability theory by introducing a categorical definition of causal models , a categorical notion of d-separation , and proving an abstract version of the d-separation criterion . This approach has two main benefits . First , categorical d-separation is a very

  • Generalization Bounds for Adversarial Contrastive Learning

    Updated: 2023-04-30 19:38:56
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Generalization Bounds for Adversarial Contrastive Learning Xin Zou , Weiwei Liu 24(114 1 54, 2023. Abstract Deep networks are well-known to be fragile to adversarial attacks , and adversarial training is one of the most popular methods used to train a robust model . To take advantage of unlabeled data , recent works have applied adversarial training to contrastive learning Adversarial Contrastive Learning ACL for short and obtain promising robust performance . However , the theory of ACL is not well understood . To fill this gap , we leverage the Rademacher omplexity to analyze the generalization

  • FLIP: A Utility Preserving Privacy Mechanism for Time Series

    Updated: 2023-04-30 19:38:56
    : Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us FLIP : A Utility Preserving Privacy Mechanism for Time Series Tucker McElroy , Anindya Roy , Gaurab Hore 24(111 1 29, 2023. Abstract Guaranteeing privacy in released data is an important goal for data-producing agencies . There has been extensive research on developing suitable privacy mechanisms in recent years . Particularly notable is the idea of noise addition with the guarantee of differential privacy . There are , however , concerns about compromising data utility when very stringent privacy mechanisms are applied . Such compromises can be quite stark in correlated data , such as time series

  • Dimensionless machine learning: Imposing exact units equivariance

    Updated: 2023-04-30 19:38:56
    : Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Dimensionless machine learning : Imposing exact units equivariance Soledad Villar , Weichi Yao , David W . Hogg , Ben Blum-Smith , Bianca Dumitrascu 24(109 1 32, 2023. Abstract Units equivariance or units covariance is the exact symmetry that follows from the requirement that relationships among measured quantities of physics relevance must obey self-consistent dimensional scalings . Here , we express this symmetry in terms of a non-compact group action , and we employ dimensional analysis and ideas from equivariant machine learning to provide a methodology for exactly units-equivariant machine

  • Concentration analysis of multivariate elliptic diffusions

    Updated: 2023-04-30 19:38:56
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Concentration analysis of multivariate elliptic diffusions Lukas Trottner , Cathrine Aeckerle-Willems , Claudia Strauch 24(106 1 38, 2023. Abstract We prove concentration inequalities and associated PAC bounds for both continuous- and discrete-time additive functionals for possibly unbounded functions of multivariate , nonreversible diffusion processes . Our analysis relies on an approach via the Poisson equation allowing us to consider a very broad class of subexponentially ergodic , multivariate diffusion processes . These results add to existing concentration inequalities for additive functionals

  • Knowledge Hypergraph Embedding Meets Relational Algebra

    Updated: 2023-04-30 19:38:56
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Knowledge Hypergraph Embedding Meets Relational Algebra Bahare Fatemi , Perouz Taslakian , David Vazquez , David Poole 24(105 1 34, 2023. Abstract Relational databases are a successful model for data storage , and rely on query languages for information retrieval . Most of these query languages are based on relational algebra , a mathematical formalization at the core of relational models . Knowledge graphs are flexible data storage structures that allow for knowledge completion using machine learning techniques . Knowledge hypergraphs generalize knowledge graphs by allowing multi-argument relations

  • Robust Load Balancing with Machine Learned Advice

    Updated: 2023-04-30 19:38:56
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Robust Load Balancing with Machine Learned Advice Sara Ahmadian , Hossein Esfandiari , Vahab Mirrokni , Binghui Peng 24(44 1 46, 2023. Abstract Motivated by the exploding growth of web-based services and the importance of efficiently managing the computational resources of such systems , we introduce and study a theoretical model for load balancing of very large databases such as commercial search engines . Our model is a more realistic version of the well-received bab model with an additional constraint that limits the number of servers that carry each piece of the data . This additional constraint

  • Sparse Training with Lipschitz Continuous Loss Functions and a Weighted Group L0-norm Constraint

    Updated: 2023-04-30 19:38:56
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Sparse Training with Lipschitz Continuous Loss Functions and a Weighted Group L0-norm Constraint Michael R . Metel 24(103 1 44, 2023. Abstract This paper is motivated by structured sparsity for deep neural network training . We study a weighted group l_0$-norm constraint , and present the projection and normal cone of this set . Using randomized smoothing , we develop zeroth and first-order algorithms for minimizing a Lipschitz continuous function constrained by any closed set which can be projected onto . Non-asymptotic convergence guarantees are proven in expectation for the proposed algorithms for

  • Benchmarking Graph Neural Networks

    Updated: 2023-04-30 19:38:56
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Benchmarking Graph Neural Networks Vijay Prakash Dwivedi , Chaitanya K . Joshi , Anh Tuan Luu , Thomas Laurent , Yoshua Bengio , Xavier Bresson 24(43 1 48, 2023. Abstract In the last few years , graph neural networks GNNs have become the standard toolkit for analyzing and learning from data on graphs . This emerging field has witnessed an extensive growth of promising techniques that have been applied with success to computer science , mathematics , biology , physics and chemistry . But for any successful field to become mainstream and reliable , benchmarks must be developed to quantify progress .

  • FedLab: A Flexible Federated Learning Framework

    Updated: 2023-04-30 19:38:56
    FedLab is a lightweight open-source framework for the simulation of federated learning. The design of FedLab focuses on federated learning algorithm effectiveness and communication efficiency. It allows customization on server optimization, client optimization, communication agreement, and communication compression. Also, FedLab is scalable in different deployment scenarios with different computation and communication resources. We hope FedLab could provide flexible APIs as well as reliable baseline implementations and relieve the burden of implementing novel approaches for researchers in the FL community. The source code, tutorial, and documentation can be found at https://github.com/SMILELab-FL/FedLab.

  • Neural Implicit Flow: a mesh-agnostic dimensionality reduction paradigm of spatio-temporal data

    Updated: 2023-04-30 19:38:56
    : Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Neural Implicit Flow : a mesh-agnostic dimensionality reduction paradigm of spatio-temporal data Shaowu Pan , Steven L . Brunton , J . Nathan Kutz 24(41 1 60, 2023. Abstract High-dimensional spatio-temporal dynamics can often be encoded in a low-dimensional subspace . Engineering applications for modeling , characterization , design , and control of such large-scale systems often rely on dimensionality reduction to make solutions computationally tractable in real time . Common existing paradigms for dimensionality reduction include linear methods , such as the singular value decomposition SVD and

  • Fitting Autoregressive Graph Generative Models through Maximum Likelihood Estimation

    Updated: 2023-04-30 19:38:56
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Fitting Autoregressive Graph Generative Models through Maximum Likelihood Estimation Xu Han , Xiaohui Chen , Francisco J . R . Ruiz , Li-Ping Liu 24(97 1 30, 2023. Abstract We consider the problem of fitting autoregressive graph generative models via maximum likelihood estimation MLE MLE is intractable for graph autoregressive models because the nodes in a graph can be arbitrarily reordered thus the exact likelihood involves a sum over all possible node orders leading to the same graph . In this work , we fit the graph models by maximizing a variational bound , which is built by first deriving the

  • Statistical Inference for Noisy Incomplete Binary Matrix

    Updated: 2023-04-30 19:38:56
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Statistical Inference for Noisy Incomplete Binary Matrix Yunxiao Chen , Chengcheng Li , Jing Ouyang , Gongjun Xu 24(95 1 66, 2023. Abstract We consider the statistical inference for noisy incomplete binary or 1-bit matrix . Despite the importance of uncertainty quantification to matrix completion , most of the categorical matrix completion literature focuses on point estimation and prediction . This paper moves one step further toward statistical inference for binary matrix completion . Under a popular nonlinear factor analysis model , we obtain a point estimator and derive its asymptotic normality .

  • Label Distribution Changing Learning with Sample Space Expanding

    Updated: 2023-04-30 19:38:56
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Label Distribution Changing Learning with Sample Space Expanding Chao Xu , Hong Tao , Jing Zhang , Dewen Hu , Chenping Hou 24(36 1 48, 2023. Abstract With the evolution of data collection ways , label ambiguity has arisen from various applications . How to reduce its uncertainty and leverage its effectiveness is still a challenging task . As two types of representative label ambiguities , Label Distribution Learning LDL which annotates each instance with a label distribution , and Emerging New Class ENC which focuses on model reusing with new classes , have attached extensive attentions .

  • Faith-Shap: The Faithful Shapley Interaction Index

    Updated: 2023-04-30 19:38:56
    : Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Faith-Shap : The Faithful Shapley Interaction Index Che-Ping Tsai , Chih-Kuan Yeh , Pradeep Ravikumar 24(94 1 42, 2023. Abstract Shapley values , which were originally designed to assign attributions to individual players in coalition games , have become a commonly used approach in explainable machine learning to provide attributions to input features for black-box machine learning models . A key attraction of Shapley values is that they uniquely satisfy a very natural set of axiomatic properties . However , extending the Shapley value to assigning attributions to interactions rather than

  • Gap Minimization for Knowledge Sharing and Transfer

    Updated: 2023-04-30 19:38:56
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Gap Minimization for Knowledge Sharing and Transfer Boyu Wang , Jorge A . Mendez , Changjian Shui , Fan Zhou , Di Wu , Gezheng Xu , Christian Gagné , Eric Eaton 24(33 1 57, 2023. Abstract Learning from multiple related tasks by knowledge sharing and transfer has become increasingly relevant over the last two decades . In order to successfully transfer information from one task to another , it is critical to understand the similarities and differences between the domains . In this paper , we introduce the notion of performance gap , an intuitive and novel measure of the distance between learning tasks

  • Sparse PCA: a Geometric Approach

    Updated: 2023-04-30 19:38:56
    : Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Sparse PCA : a Geometric Approach Dimitris Bertsimas , Driss Lahlou Kitane 24(32 1 33, 2023. Abstract We consider the problem of maximizing the variance explained from a data matrix using orthogonal sparse principal components that have a support of fixed cardinality . While most existing methods focus on building principal components PCs iteratively through deflation , we propose GeoSPCA , a novel algorithm to build all PCs at once while satisfying the orthogonality constraints which brings substantial benefits over deflation . This novel approach is based on the left eigenvalues of the covariance

  • Decentralized Learning: Theoretical Optimality and Practical Improvements

    Updated: 2023-04-30 19:38:56
    : Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Decentralized Learning : Theoretical Optimality and Practical Improvements Yucheng Lu , Christopher De Sa 24(93 1 62, 2023. Abstract Decentralization is a promising method of scaling up parallel machine learning systems . In this paper , we provide a tight lower bound on the iteration complexity for such methods in a stochastic non-convex setting . Our lower bound reveals a theoretical gap in known convergence rates of many existing decentralized training algorithms , such as D-PSGD . We prove by construction this lower bound is tight and achievable . Motivated by our insights , we further propose

  • Non-Asymptotic Guarantees for Robust Statistical Learning under Infinite Variance Assumption

    Updated: 2023-04-30 19:38:56
    There has been a surge of interest in developing robust estimators for models with heavy-tailed and bounded variance data in statistics and machine learning, while few works impose unbounded variance. This paper proposes two types of robust estimators, the ridge log-truncated M-estimator and the elastic net log-truncated M-estimator. The first estimator is applied to convex regressions such as quantile regression and generalized linear models, while the other one is applied to high dimensional non-convex learning problems such as regressions via deep neural networks. Simulations and real data analysis demonstrate the robustness of log-truncated estimations over standard estimations.

  • Labels, Information, and Computation: Efficient Learning Using Sufficient Labels

    Updated: 2023-04-30 19:38:56
    , , : Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Labels , Information , and Computation : Efficient Learning Using Sufficient Labels Shiyu Duan , Spencer Chang , Jose C . Principe 24(31 1 35, 2023. Abstract In supervised learning , obtaining a large set of fully-labeled training data is expensive . We show that we do not always need full label information on every single training example to train a competent classifier . Specifically , inspired by the principle of sufficiency in statistics , we present a statistic a summary of the fully-labeled training set that captures almost all the relevant information for classification but at the same

  • Outlier-Robust Subsampling Techniques for Persistent Homology

    Updated: 2023-04-30 19:38:56
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Outlier-Robust Subsampling Techniques for Persistent Homology Bernadette J . Stolz 24(90 1 35, 2023. Abstract In recent years , persistent homology has been successfully applied to real-world data in many different settings . Despite significant computational advances , persistent homology algorithms do not yet scale to large datasets preventing interesting applications . One approach to address computational issues posed by persistent homology is to select a set of landmarks by subsampling from the data . Currently , these landmark points are chosen either at random or using the maxmin algorithm .

  • Neural Operator: Learning Maps Between Function Spaces With Applications to PDEs

    Updated: 2023-04-30 19:38:56
    : Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Neural Operator : Learning Maps Between Function Spaces With Applications to PDEs Nikola Kovachki , Zongyi Li , Burigede Liu , Kamyar Azizzadenesheli , Kaushik Bhattacharya , Andrew Stuart , Anima Anandkumar 24(89 1 97, 2023. Abstract The classical development of neural networks has primarily focused on learning mappings between finite dimensional Euclidean spaces or finite sets . We propose a generalization of neural networks to learn operators , termed neural operators , that map between infinite dimensional function spaces . We formulate the neural operator as a composition of linear integral

  • HiClass: a Python Library for Local Hierarchical Classification Compatible with Scikit-learn

    Updated: 2023-04-30 19:38:56
    : Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us HiClass : a Python Library for Local Hierarchical Classification Compatible with Scikit-learn Fábio M . Miranda , Niklas Köhnecke , Bernhard Y . Renard 24(29 1 17, 2023. Abstract HiClass is an open-source Python library for local hierarchical classification entirely compatible with scikit-learn . It contains implementations of the most common design patterns for hierarchical machine learning models found in the literature , that is , the local classifiers per node , per parent node and per level . Additionally , the package contains implementations of hierarchical metrics , which are more

  • Dimension-Grouped Mixed Membership Models for Multivariate Categorical Data

    Updated: 2023-04-30 19:38:56
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Dimension-Grouped Mixed Membership Models for Multivariate Categorical Data Yuqi Gu , Elena E . Erosheva , Gongjun Xu , David B . Dunson 24(88 1 49, 2023. Abstract Mixed Membership Models MMMs are a popular family of latent structure models for complex multivariate data . Instead of forcing each subject to belong to a single cluster , MMMs incorporate a vector of subject-specific weights characterizing partial membership across clusters . With this flexibility come challenges in uniquely identifying , estimating , and interpreting the parameters . In this article , we propose a new class of

  • Gaussian Processes with Errors in Variables: Theory and Computation

    Updated: 2023-04-30 19:38:56
    : Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Gaussian Processes with Errors in Variables : Theory and Computation Shuang Zhou , Debdeep Pati , Tianying Wang , Yun Yang , Raymond J . Carroll 24(87 1 53, 2023. Abstract Covariate measurement error in nonparametric regression is a common problem in nutritional epidemiology and geostatistics , and other fields . Over the last two decades , this problem has received substantial attention in the frequentist literature . Bayesian approaches for handling measurement error have only been explored recently and are surprisingly successful , although there still is a lack of a proper theoretical

  • The SKIM-FA Kernel: High-Dimensional Variable Selection and Nonlinear Interaction Discovery in Linear Time

    Updated: 2023-04-30 19:38:56
    : Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us The SKIM-FA Kernel : High-Dimensional Variable Selection and Nonlinear Interaction Discovery in Linear Time Raj Agrawal , Tamara Broderick 24(27 1 60, 2023. Abstract Many scientific problems require identifying a small set of covariates that are associated with a target response and estimating their effects . Often , these effects are nonlinear and include interactions , so linear and additive methods can lead to poor estimation and variable selection . Unfortunately , methods that simultaneously express sparsity , nonlinearity , and interactions are computationally intractable with runtime at

  • Generalization Bounds for Noisy Iterative Algorithms Using Properties of Additive Noise Channels

    Updated: 2023-04-30 19:38:56
    Machine learning models trained by different optimization algorithms under different data distributions can exhibit distinct generalization behaviors. In this paper, we analyze the generalization of models trained by noisy iterative algorithms. We derive distribution-dependent generalization bounds by connecting noisy iterative algorithms to additive noise channels found in communication and information theory. Our generalization bounds shed light on several applications, including differentially private stochastic gradient descent (DP-SGD), federated learning, and stochastic gradient Langevin dynamics (SGLD). We demonstrate our bounds through numerical experiments, showing that they can help understand recent empirical observations of the generalization phenomena of neural networks.

  • Discrete Variational Calculus for Accelerated Optimization

    Updated: 2023-04-30 19:38:56
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Discrete Variational Calculus for Accelerated Optimization Cédric M . Campos , Alejandro Mahillo , David Martín de Diego 24(25 1 33, 2023. Abstract Many of the new developments in machine learning are connected with gradient-based optimization methods . Recently , these methods have been studied using a variational perspective Betancourt et al . 2018 This has opened up the possibility of introducing variational and symplectic methods using geometric integration . In particular , in this paper , we introduce variational integrators Marsden and West , 2001 which allow us to derive different methods for

  • Bayes-Newton Methods for Approximate Bayesian Inference with PSD Guarantees

    Updated: 2023-04-30 19:38:56
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Bayes-Newton Methods for Approximate Bayesian Inference with PSD Guarantees William J . Wilkinson , Simo Särkkä , Arno Solin 24(83 1 50, 2023. Abstract We formulate natural gradient variational inference VI expectation propagation EP and posterior linearisation PL as extensions of Newton's method for optimising the parameters of a Bayesian posterior distribution . This viewpoint explicitly casts inference algorithms under the framework of numerical optimisation . We show that common approximations to Newton's method from the optimisation literature , namely Gauss-Newton and quasi-Newton methods e.g .

  • Calibrated Multiple-Output Quantile Regression with Representation Learning

    Updated: 2023-04-30 19:38:56
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Calibrated Multiple-Output Quantile Regression with Representation Learning Shai Feldman , Stephen Bates , Yaniv Romano 24(24 1 48, 2023. Abstract We develop a method to generate predictive regions that cover a multivariate response variable with a user-specified probability . Our work is composed of two components . First , we use a deep generative model to learn a representation of the response that has a unimodal distribution . Existing multiple-output quantile regression approaches are effective in such cases , so we apply them on the learned representation , and then transform the solution to

  • Fast Online Changepoint Detection via Functional Pruning CUSUM Statistics

    Updated: 2023-04-30 19:38:56
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Fast Online Changepoint Detection via Functional Pruning CUSUM Statistics Gaetano Romano , Idris A . Eckley , Paul Fearnhead , Guillem Rigaill 24(81 1 36, 2023. Abstract Many modern applications of online changepoint detection require the ability to process high-frequency observations , sometimes with limited available computational resources . Online algorithms for detecting a change in mean often involve using a moving window , or specifying the expected size of change . Such choices affect which changes the algorithms have most power to detect . We introduce an algorithm , Functional Online CuSUM

  • Approximate Post-Selective Inference for Regression with the Group LASSO

    Updated: 2023-04-30 19:38:56
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Approximate Post-Selective Inference for Regression with the Group LASSO Snigdha Panigrahi , Peter W MacDonald , Daniel Kessler 24(79 1 49, 2023. Abstract After selection with the Group LASSO or generalized variants such as the overlapping , sparse , or standardized Group LASSO inference for the selected parameters is unreliable in the absence of adjustments for selection bias . In the penalized Gaussian regression setup , existing approaches provide adjustments for selection events that can be expressed as linear inequalities in the data variables . Such a representation , however , fails to hold

  • A Likelihood Approach to Nonparametric Estimation of a Singular Distribution Using Deep Generative Models

    Updated: 2023-04-30 19:38:56
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us A Likelihood Approach to Nonparametric Estimation of a Singular Distribution Using Deep Generative Models Minwoo Chae , Dongha Kim , Yongdai Kim , Lizhen Lin 24(77 1 42, 2023. Abstract We investigate statistical properties of a likelihood approach to nonparametric estimation of a singular distribution using deep generative models . More specifically , a deep generative model is used to model high-dimensional data that are assumed to concentrate around some low-dimensional structure . Estimating the distribution supported on this low-dimensional structure , such as a low-dimensional manifold , is

  • A Randomized Subspace-based Approach for Dimensionality Reduction and Important Variable Selection

    Updated: 2023-04-30 19:38:56
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us A Randomized Subspace-based Approach for Dimensionality Reduction and Important Variable Selection Di Bo , Hoon Hwangbo , Vinit Sharma , Corey Arndt , Stephanie TerMaath 24(76 1 31, 2023. Abstract An analysis of high-dimensional data can offer a detailed description of a system but is often challenged by the curse of dimensionality . General dimensionality reduction techniques can alleviate such difficulty by extracting a few important features , but they are limited due to the lack of interpretability and connectivity to actual decision making associated with each physical variable . Variable

  • Intrinsic Persistent Homology via Density-based Metric Learning

    Updated: 2023-04-30 19:38:56
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Intrinsic Persistent Homology via Density-based Metric Learning Ximena Fernández , Eugenio Borghini , Gabriel Mindlin , Pablo Groisman 24(75 1 42, 2023. Abstract We address the problem of estimating topological features from data in high dimensional Euclidean spaces under the manifold assumption . Our approach is based on the computation of persistent homology of the space of data points endowed with a sample metric known as Fermat distance . We prove that such metric space converges almost surely to the manifold itself endowed with an intrinsic metric that accounts for both the geometry of the

  • Inference for a Large Directed Acyclic Graph with Unspecified Interventions

    Updated: 2023-04-30 19:38:56
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Inference for a Large Directed Acyclic Graph with Unspecified Interventions Chunlin Li , Xiaotong Shen , Wei Pan 24(73 1 48, 2023. Abstract Statistical inference of directed relations given some unspecified interventions i.e . the intervention targets are unknown is challenging . In this article , we test hypothesized directed relations with unspecified interventions . First , we derive conditions to yield an identifiable model . Unlike classical inference , testing directed relations requires identifying the ancestors and relevant interventions of hypothesis-specific primary variables . To this end

  • Distributed Nonparametric Regression Imputation for Missing Response Problems with Large-scale Data

    Updated: 2023-04-30 19:38:56
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Distributed Nonparametric Regression Imputation for Missing Response Problems with Large-scale Data Ruoyu Wang , Miaomiao Su , Qihua Wang 24(68 1 52, 2023. Abstract Nonparametric regression imputation is commonly used in missing data analysis . However , it suffers from the curse of dimension . The problem can be alleviated by the explosive sample size in the era of big data , while the large-scale data size presents some challenges in the storage of data and the calculation of estimators . These challenges make the classical nonparametric regression imputation methods no longer applicable . This

  • Prior Specification for Bayesian Matrix Factorization via Prior Predictive Matching

    Updated: 2023-04-30 19:38:56
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Prior Specification for Bayesian Matrix Factorization via Prior Predictive Matching Eliezer de Souza da Silva , Tomasz Kuśmierczyk , Marcelo Hartmann , Arto Klami 24(67 1 51, 2023. Abstract The behavior of many Bayesian models used in machine learning critically depends on the choice of prior distributions , controlled by some hyperparameters typically selected through Bayesian optimization or cross-validation . This requires repeated , costly , posterior inference . We provide an alternative for selecting good priors without carrying out posterior inference , building on the prior predictive

  • Wide-minima Density Hypothesis and the Explore-Exploit Learning Rate Schedule

    Updated: 2023-04-30 19:38:56
    Several papers argue that wide minima generalize better than narrow minima. In this paper, through detailed experiments that not only corroborate the generalization properties of wide minima, we also provide empirical evidence for a new hypothesis that the density of wide minima is likely lower than the density of narrow minima. Further, motivated by this hypothesis, we design a novel explore-exploit learning rate schedule. On a variety of image and natural language datasets, compared to their original hand-tuned learning rate baselines, we show that our explore-exploit schedule can result in either up to 0.84% higher absolute accuracy using the original training budget or up to 57% reduced training time while achieving the original reported accuracy.

  • Fundamental limits and algorithms for sparse linear regression with sublinear sparsity

    Updated: 2023-04-30 19:38:56
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Fundamental limits and algorithms for sparse linear regression with sublinear sparsity Lan V . Truong 24(64 1 49, 2023. Abstract We establish exact asymptotic expressions for the normalized mutual information and minimum mean-square-error MMSE of sparse linear regression in the sub-linear sparsity regime . Our result is achieved by a generalization of the adaptive interpolation method in Bayesian inference for linear regimes to sub-linear ones . A modification of the well-known approximate message passing algorithm to approach the MMSE fundamental limit is also proposed , and its state evolution is

  • Topological Convolutional Layers for Deep Learning

    Updated: 2023-04-30 19:38:56
    This work introduces the Topological CNN (TCNN), which encompasses several topologically defined convolutional methods. Manifolds with important relationships to the natural image space are used to parameterize image filters which are used as convolutional weights in a TCNN. These manifolds also parameterize slices in layers of a TCNN across which the weights are localized. We show evidence that TCNNs learn faster, on less data, with fewer learned parameters, and with greater generalizability and interpretability than conventional CNNs. We introduce and explore TCNN layers for both image and video data. We propose extensions to 3D images and 3D video.

  • Necessary and Sufficient Conditions for Inverse Reinforcement Learning of Bayesian Stopping Time Problems

    Updated: 2023-04-30 19:38:56
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Necessary and Sufficient Conditions for Inverse Reinforcement Learning of Bayesian Stopping Time Problems Kunal Pattanayak , Vikram Krishnamurthy 24(52 1 64, 2023. Abstract This paper presents an inverse reinforcement learning IRL framework for Bayesian stopping time problems . By observing the actions of a Bayesian decision maker , we provide a necessary and sufficient condition to identify if these actions are consistent with optimizing a cost function . In a Bayesian partially observed setting , the inverse learner can at best identify optimality wrt the observed strategies . Our IRL algorithm

  • A Group-Theoretic Approach to Computational Abstraction: Symmetry-Driven Hierarchical Clustering

    Updated: 2023-04-30 19:38:56
    : Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us A Group-Theoretic Approach to Computational Abstraction : Symmetry-Driven Hierarchical Clustering Haizi Yu , Igor Mineyev , Lav R . Varshney 24(47 1 61, 2023. Abstract Humans' abstraction ability plays a key role in concept learning and knowledge discovery . This theory paper presents the mathematical formulation for computationally emulating human-like abstractions---computational abstraction---and abstraction processes developed hierarchically from innate priors like symmetries . We study the nature of abstraction via a group-theoretic approach , formalizing and practically computing abstractions

  • ✚ Visualization Tools and Learning Resources – April 2023 Roundup

    Updated: 2023-04-27 18:30:47
    Membership Courses Tutorials Projects Newsletter Become a Member Log in Members Only Visualization Tools and Learning Resources April 2023 Roundup April 27, 2023 Topic The Process roundup Welcome to The Process where we look closer at how the charts get made . I’m Nathan Yau , and every month I collect visualization tools and resources to help you make better charts . Here is the good stuff for . April To access this issue of The Process , you must be a . member If you are already a member , log in here See What You Get The Process is a weekly newsletter where I evaluate how visualization tools , rules , and guidelines work in practice . I publish every Thursday . Get it in your inbox or access it via the site . You also gain unlimited access to hundreds of hours worth of step-by-step

  • Nick Mar Uses AnyChart JS Charts to Visualize User Performance and Stock Market Data

    Updated: 2023-04-24 11:06:22
    As the world’s top provider of cutting-edge data visualization solutions, we are constantly thrilled to witness how our products are utilized by both businesses and individuals to create interactive charts and dashboards. We recently had the opportunity to chat with software developer Nick Mar, who shared some of his personal projects with us, highlighting the […] The post Nick Mar Uses AnyChart JS Charts to Visualize User Performance and Stock Market Data appeared first on AnyChart News.

  • ✚ Explaining More, Assuming Less

    Updated: 2023-04-20 18:30:35
    , Membership Courses Tutorials Projects Newsletter Become a Member Log in Members Only Explaining More , Assuming Less April 20, 2023 Topic The Process Visualization assumptions audience teaching Welcome to The Process where we look closer at how the charts get made . I’m Nathan Yau , and I’m figuring out what to assume from readers , even if they are smart and . well-informed To access this issue of The Process , you must be a . member If you are already a member , log in here See What You Get The Process is a weekly newsletter where I evaluate how visualization tools , rules , and guidelines work in practice . I publish every Thursday . Get it in your inbox or access it via the site . You also gain unlimited access to hundreds of hours worth of step-by-step visualization courses and

  • Qlik Gantt & Sunburst’s Exciting New Features + Upgrades Across All Extensions!

    Updated: 2023-04-18 15:10:49
    Sales : 1 888 845-1211 USA or 44 20 7193 9444 Europe customer login Toggle navigation Products AnyChart AnyStock AnyMap AnyGantt Mobile Qlik Extension Features Resources Business Solutions Technical Integrations Chartopedia Tutorials Support Company About Us Customers Success Stories More Testimonials News Download Buy Now Search News » News » Qlik Gantt Sunburst’s Exciting New Features Upgrades Across All Extensions Qlik Gantt Sunburst’s Exciting New Features Upgrades Across All Extensions April 18th , 2023 by AnyChart Team Hey Qlikkies We have some exciting news to share with you today . AnyChart’s  Qlik Sense Extensions  have undergone a major update as we’ve made improvements across all of them , including the  Decomposition Tree However , we are particularly thrilled to announce

  • Join AnyChart at QlikWorld 2023: Sponsor & Exhibitor

    Updated: 2023-04-14 13:27:06
    Get ready, because QlikWorld 2023 is coming in hot! From April 17th to 20th, Las Vegas will be bustling with all the latest and greatest in the world of data analytics, and AnyChart is pumped to be part of the action. Meet us in Booth #180! Read more at qlik.anychart.com » The post Join AnyChart at QlikWorld 2023: Sponsor & Exhibitor appeared first on AnyChart News.

  • Building 3D Surface Plot in JavaScript

    Updated: 2023-04-11 18:24:21
    Welcome to this tutorial on creating a visually stunning and interactive 3D surface plot using JavaScript! If you’re passionate about data visualization and want to expand your skills to the next level, this guide is for you. Here, we’ll take you through a step-by-step process to create an engaging 3D surface plot that will make […] The post Building 3D Surface Plot in JavaScript appeared first on AnyChart News.

Current Feed Items | Previous Months Items

Mar 2023 | Feb 2023 | Jan 2023 | Dec 2022 | Nov 2022